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(57) Abstract: Disclosed are methods and apparatus 
for communicating over a computer network through 
the intuitive placement and movement of one or more 
physical objects. In general terms, a sensor and one or 
more physical objects are provided. The sensor may 
form part of a platform upon which objects are placed 
or the sensor may be located proximate to such a plat- 
form. When one or more of the physical objects are 
moved over particular portions of the platform, data 
content may be accessed from a remote site depending 
on the placement or movement of such objects. The 
data content may be displayed as part of a particular 
application associated with the platform and/or physi- 
cal objects. In a specific implementation, the platform 
represents a game board and the physical objects are 
game pieces for moving over the game board. When 
the game pieces are moved to particular board loca- 
tions, game modules are downloaded to and executed 
on the user's local computer game system (e.g., on a 
computer display or television screen). In another im- 
plementadon, particular web pages that enhance the 
game experience are accessed and displayed for the 
user when the user moves one or more game pieces 
over a particular board location. 
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Remote Data Access Through Manipulation of Physical 

Objects 

By Inventors : 

Marcos R. Vescovi, David Kumar, John Sakalowsky, and Adam 
^ C. Jordan 

Cross Reference to Related Applications 

This application claims priority to U.S. Provisional Application No. 
60168,474 by Vescovi et al., filed 1 December 1999, entitled INTER^^{ET ACCESS 
THROUGH MANIPULATION OF ONE OR MORE PHYSICAL OBJECTS. This 

10 application is also related to (1) U.S. Patent No. 5,953,686 by Piernot et al., issued 14 
September 1999, entitled VIDEO CAMERA BASED COMPUTER INPUT 
SYSTEM WITH INTERCHANGEABLE PHYSICAL INTERFACE, (2) U.S. 
Patent Application No. 09/018,023 by Piernot et al., filed 2 February 1998, entitled 
COMPUTER METHOD AND APPARATUS FOR INTERACTING WITH A 

1 5 PHYSICAL SYSTEM, (3) U.S. Patent Application No. 09/144,951 by Vescovi et al., 
filed 1 September 1998, entitled DETECTING PHYSICAL OBJECTS STATES 
USING ELECTROMAGNETIC SENSORS. (4) US. Patent Application No. 
08/017,450 by Vescovi et al., filed 2 February 1998, entitled CODED OBJECT 
SYSTEM AND CODE RECOGNITION METHODS. (5) U.S. Patent 6,108.612 by 

20 Vescovi et al.. issued 22 August 2000. entitled CODED OBJECTS AND 
METHODS FOR DETECTING SUCH CODED OBJECTS. These patent 
applications are herein incorporated by reference in their entirety. 

Background of the Invention 

The present invention pertains to apparatus and methods for remotely 
'■ 5 accessing data through a network. 
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The digital living room with the central television set has been the target of 
many content delivery vehicles and content providers. Currently, there are over four 
hundred cable channels. Even more content is being offered by satellite channels. 
Most recently, web access has been added to television content delivery systems. 
With the addition of the web, potentially millions of web pages are accessible to the 
user through his/her television Set top box equipment scheduled to be introduced 
soon, along with the new generation of game consoles, are also equipped to provide 
access to significantly more content. 

As the WEB and cable television begin to contain a large number of content 
varieties and subjects, increasingly, the user finds locating a particular content 
segment (e.g.. channel or web page) difficult. In other words, the user typically finds 
It difficult to sift through such a vast array of choices. Traditional remote controls 
provided with cable boxes and television sets are cumbersome tools for searching 
through content On screen menu system are increasingly becoming too crowded 
and unuseable as the number of content choices rapidly increases. 

Though the variety and size of content has grown, the user has been unable to 
access such content with the ease of the traditional remote control unit and television 
network system Several remote controls boast increasing numbers of option 
selection features and numerous buttons. However these additions have not made the 
interaction easier in any way. Navigating through the myriad of channels and time 
slots using buttons and second-function buttons has become a complex exercise. The 
introduction of WEB content through equipment such as the WEBTV has made it 
even more difficult to use a remote control to access content. 

Additionally, recording fiiture programs has become increasingly difficult in 
the typical household with several remote control units and the high number of 
available channels. Even the universal remote controls cannot seem to tackle the 
task any better. The users have largely ignored the introduction of VCR+ features 
with its need for complicated initial connections necessary to interconnect cable, TV 
and VCR equipment 
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There are, of course, a number of human/computer interfaces which allow 
users, with van/ing degrees of comfort and ease, to interact with computers. For 
example, keyboards, computer mice, joysticks, etc. allow users to physically 
manipulate a single three-dimensional object (e.g., by typing or pointing and 
clicking) to search through remote content However, these human/computer 
interfaces are quite aaificial in nature, and tend to require a substantial investment in 
training to be used efficiently. The artificial metaphors tend to break down and 
systems such as keyboards and mice tend to have inherently low rates of data input. 

Accordingly, a simpler mechanism for accessing remote content is desired. 
Such mechanisms would also preferably provide ready access to content using 
natural intuitive motions. 



Summary of the Invention 

Accordingly, the present invention provides methods and apparatus for 
communicating over a computer network through the intuitive placement and 
15 movement of one or more physical objects In general terms, a sensor and one or 
more physical objects are provided. The sensor may form part of a platform upon 
which objects are placed or the sensor may be located proximate to such a platform. 
When one or more of the physical objects are moved over particular portions of the 
platform, data content may be accessed from a remote site depending on the 
2 0 placement or movement of such objects. The data content may be displayed as part 
of a particular application associated with the platform and/or physical objects In a 
specific implementation, the platform represents a game board and the physical 
objects are game pieces for moving over the game board. When the game pieces are 
moved to particular board locations, game modules are downloaded to and executed 
25 on the user's local computer game system (e.g., on a computer display or television 
screen) In another implementation, particular web pages that enhance the game 
experience are accessed and displayed for the user when the user moves one or more 
game pieces over a particular board location. 

In one embodiment, an interactive system associated with a first computer for 
SUBSTITUTE SHEET (RULE 26) 
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communicating with a second computer over a wide area network is disclosed. The 
interactive system includes a detection system having a plurality of physical objects 
and a sensor. The physical objects have a plurality of states that are each detectable 
by the sensor. The interactive system further includes a computer readable medium 
having programming instructions for communicating with the detection system and 
the second computer Communication with the second computer is based on a 
combination of two or more states of the physical objects detected. 

In another embodiment, the interactive system has a detection system having 
a physical object and a sensor. The physical object has a plurality of states that are 
each detectable by the sensor and that are states other than position states of the 
physical object relative to the sensor. The mteractive system also mcludes a 
computer readable medium having programming instructions for communicating 
with the detection system via the first computer and for communicating with the 
second computer via a wide area network. Communication with the second 
computer network is based on a current state of the physical object. 

In an alternative embodiment, the interactive system includes a detection 
system having a physical object and a sensor, and the physical object has a plurality 
of orientation positions that are each detectable by the sensor. The imeractive system 
also has a computer readable medium having programming instructions for 
2 0 communicating with the detection system via the first computer and the second 
computer via a wide area network (e.g., the Internet). Communication with the 
second computer is based on a current orientation of the physical object. 

In yet another alternative embodiment, the interactive system includes a 
detection system having a plurality of physical objects and a sensor. The physical 

2 5 objects have a plurality of positions that are each detectable by the sensor. The 

interactive system also includes a computer readable medium having programming 
instructions for communicating with the detection system and the second computer. 
Communication with the second computer is based on one or more positions of the 
physical objects, wherein the positions are detected by the sensor and are non- 

3 0 discrete. A method for using such a system is also disclosed. 
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In yet another embodiment, the interactive system has a detection system 
having a plurality of physical objects and a sensor. Interaction data associated with 
the physical objects is detectable by the sensor. The interactive system further 
includes a computer readable medium having programming instructions for 
communicating with the detection system and the second computer. Communication 
with the second computer is based on detected interaction data associated with one or 
more of the physical objects. The communication is in the form of either 
downloading game content from the second computer to the first computer, posting 
game content from the first computer to the second computer, displaying a web page 
at the first cpmputer, initiating a chat session between the first and second computers, 
sending or receiving an email message to or from the second computer, setting up a 
video conference, or streaming video or audio to the first computer from the second 
computer or to the second computer fi-om the first computer. A method for using 
such a system is also disclosed. 

These and other features and advantages of the present invention will be 
presented in more detail in the following specification of the invention and the 
accompanying figures which illustrate by way of example the principles of the 
invention. 



Brief Description of the Drawings 

FIG. 1 is a diagrammatic representation of an interactive content access 
system in accordance with one embodiment of the present invention. 

FIG. 2 is a flowchart illustrating a process of utilizing an interacfive content 
access system, such as the system in FIG. 1, in accordance with one embodiment of 
the present invention. 

FIG. 3 is a flow chart illustrating the operation of FIG. 2 of interrupting the 
extracted data in accordance with one embodiment of the present invention. 

FIG. 4 is a flow chart illustrating the operation of FIG. 2 of executing an 
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application program based on data associated with one or more physical objects in 
accordance with one embodiment of the present invention. 

FIG. 5 IS a diagrammatic representation of an interactive television system in 
accordance with an alternative embodiment of the present invention. 



Detailed Description of Specific Embodiments 

Reference will now be made in detail to the specific embodiments of the 
invention. Examples of the these specific embodiments are illustrated in the 
accompanying drawings While the invention will be described in conjunction with 
these specific embodiments, it will be understood that it is not intended to limit the 
invention to the described embodiments. On the contrary, it is intended to cover 
alternatives, modifications, and equivalents as may be included within the spirit and 
scope of the invention as defined by the appended claims. In the following 
description, numerous specific details are set forth in order to provide a thorough 
understanding of the present invention The present invention may be practiced 
without some or all of these specific details In other instances, well known process 
operations have not been described in detail in order not to unnecessarily obscure the 
present invention. 

In general terms, the present invention provides mechanisms for a local 
computer system to access data content from a remote site or send data content to a 
remote site by manipulating one or more physical objects within a physical system or 
landscape in communication with the local computer system. The physical objects 
and/or landscape may take any suitable form Preferably, the physical objects and 
landscape symbolically represent the type of data content being accessed or sent. For 
example, the objects may represent characters that are moved over a game board 
coupled to a computer or television upon which a computer adventure game is 
presented. In a specific example, new content ;s retrieved and displayed when a 
character object is moved to a symbol on the boerd that represents or depicts some 
form of transportation device, such as a teiepon pad. In this example, the user's 



SUBSTITUTE SHEET (RULE 26) 



wo 01/40921 PCT/DKOO/00662 



game character is then transported to a new landscape or world (e.g., new game 
content is downloaded), which new world is displayed on the user's computer screen. 
In this embodiment an intuitive physical system is provided to allow easy access to 
on-line activities. 

Any suitable data content may be accessed from a remote site based on 
positional and/or identity data of one or more physical objects. By way of general 
examples, data content accesses may include hypertext transfers, text or graphical 
file transfers {e.g.. via ftp), and email transfers from a remote site to the local user 
site In the above game application, as game pieces are moved over a game board, 
new game modules or content may be automatically downloaded into the computer 
game system to provide additional game features (e.g., additional character abilities, 
additional game interactions, and addhional game characters). For example, 
additional game scenarios may be downloaded from a remote site into the local game 
system for an adventure type game. In another example, the objects may represent 
"brick and mortar" buildings that activate corresponding business web sites when 
placed on a sensing platform coupled to a computer system. For instance, when a 
building piece that resembles a pet shop is placed within a particular spot on the 
platform, a corresponding on-line pet shop's web page is displayed on the user's 
computer system display. Alternatively, when a character piece moves next to a 
building piece or onto a building symbol on the platform, a web page that 
corresponds to the building is displayed. 

Data content may also be posted or transmitted from the local system to 
another remote site through the manipulation of physical objects In the game 
example, the user may post game content through the placement or movement of 
game pieces or physical objects In another example, an email is sent from the user 
site to a remote site when one or more physical objects are arranged or moved in 
particular configurations. 

Communication between the local computer system and one or more remote 
computer systems may also be initiated through the physical objects. For example, 
multiple user play may be initiated through the orientation of physical objects 
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relative to one another or the location of one or more physical objects relative to a 
game platform. For instance, game characters corresponding to remote players may 
be added to the local user's game display. Other communication types include email 
exchanges, chat sessions, video and/or audio streaming, and video conferencing. For 
example, game players may exchange statistic cards for their game characters in an 
adventure type game or dollhouse wallpaper for their doUhouse design game. 

FIG. 1 shows an interactive system 100 in accordance with one embodiment 
of the present invention The interactive system 100 is configured to provide remote 
access to data content (as well as local content) based on detected positions and/or 
identities of one or more physical objects The interactive system 100 includes a 
detection system 103 and a computer system 110. The detection system generally 
detects positional and/or identity data from one or more physical objects and 
communicates the detected data to the computer system. The computer system 110 
generally analyzes the detected data and selects which data content (if any) to access 
based on the detected data The data content may reside locally on computer system 
1 10 (eg,, in application software 112) or reside on a remote computer system (not 
shown) accessible through a network interface 121. The data content form depends 
on the particular application (e.g., game content for a game application). 

The detection system 103 may include any suitable mechanisms for detecting 
the presence, positions, and identities of the various objects (e.g., 106). In the 
illustrated embodiment of FIG. 1, the detection system 103 is in the form of an 
electromagnetic detection system for detecting the objects and inputting the detected 
data to the computer system 110. Any suitable electromagnetic detection system 
may be implemented. Several examples of electromagnetic detection systems that 
work well with the present invention are described in PCT Application No. 
PCT/GB98/01759 (W098/58237(A1 )) and US Patent 5,815,091, which are herein 
incorporated by reference in their entirety. 

As shown, the detection system 103 includes an I/O board 108, a printed 
circuit (PC) board 101 having an antenna 102, a platform 104, and one or more 
objects (e.g., 106 and 109). F;iG. 1 includes an exploded perspective of the printed 
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board and platform so as to more clearing illustrate these pans. One or more of the 
objects each include a resonator. In one implementation, each resonator is in the 
form of a coil in series with a capacitor, which resonates at a particular resonating 
frequency. An excitation signals may be applied to antenna 102 to excite a 
particularly configured resonator of a particular object. In response to the excitation 
signal, a resonating signal from the object that has the same frequency as the 
excitation signal and that is located proximate to the antenna 102 is sensed by the 
antenna 102 The platform 104 is placed relative to the antenna 102 so that objects 
placed near the platform are detectable by the antenna 102. 

The platform 104 may be a visual representation of the particular application 
software 1 12. For example, the platform 104 may be in the form of a game board for 
a particular game application software. Of course, the detection system does not 
require a platform That is, the objects may simply be placed proximate to the 
printed board 101 and antenna 102 embedded therein. 

The antenna 102 may be any suitable form for sensing positions of the 
objects. In a specific implementation, the antenna 102 is in the form of four 
transmission coils and four reception coils. The transmission coils are configured to 
approximate multiple period sin and cos fields in the x direction, as well as multiple 
period sin and cos fields in the y direction. Likewise, the reception coils 
approximate multiple period sin and cos fields in both the x and y directions. 
Preferably, the sin and cos reception coils each have a different number of periods to 
facilitate determination of a position of a resonator along such coils. 

The computer system 110 may include, among other things, a driver 114, 
application software 112, interactive framework software 120, display 116, and 
speakers 118. Several well known components of a computer system, such as a 
processor and memory unit, are not described or included within FIG. 1 so as to not 
obscure the invention. 

The I/O board 108 provides an interface between the detection system 103 
and computer system 110. In one embodiment, the I/O board passes signals for 
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controlling the other components of the detection system (e,g,, the antenna 102 and 
PC board 101) from the computer system 110 to the detection system components 
and passes detected data signals from the one or more physical objects to the 
computer system 110. 

Application software 1 12 generally controls the detection system 103 via I/O 
board 108. The application software controls what signals are transmitted over the 
transmission coils and analyzes signals received on the reception coils in response to 
the transmitted signals. Although the application software 112 is shown as residing 
only on the computer system 110, of course, it may also reside elsewhere {e.g., 
within memory of the I/O Board or the PC Board) Aiiernatively, a portion of the 
application may also be downloaded to a microprocessor within the PC board 101 
(not shown). 

The application software may be configured to select specific objects to be 
detected by the detection mechanism In other words, the software selects which 
15 object is to be polled. The software also selects a frequency that corresponds to the 
resonating frequency of the selected object. Alternatively, the specific objects may 
output coded signals that identify the corresponding objects. The application 
software then analyzes the coded signals to determine an identity and position of 
each of the corresponding objects. 

2 0 In the later example, after a particular object and its corresponding frequency 

are selected, the software then causes an excitation signal having the selected 
frequency to be sequentially generated on each of the transmission coils (sinx, cosx, 
siny, cosy) Each of the transmitted excitation signals is in the form of an AC current 
having the selected frequency that causes an AC field over each of the transmission 

2 5 coils. 

If present, the resonator of the corresponding object resonates in response to 
the AC field of each excitation signal, thereby inducing an AC field in the resonator. 
The AC field of the resonator couples with corres{)onding reception coils to induce a 
resonator signal on the reception coils. In effect, :he resonator of the object reflects 
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the excitation signals onto the corresponding reception coils in the form of a 
resonator signal. 

Any suitable mechanisms may be implemented to convert the resonator or 
detected signals into signals that are appropriate for input into a computer system. 
As shown, the detected signal is sent through the I/O board 108 through driver 1 14 to 
application program 112. The I/O board 108 may be arranged to convert a detected 
signal to a digital signal (e g., a binan,' pulse wave) that corresponds to an average 
voltage value. 

In a specific implementation, the average voltage of each resonator signal 
from each reception coil may be obtained by amplifying, mixing, and filtering each 
resonator signal. This signal processing may be performed by any suitable hardware 
and/or software components (c.^'.. located within the PC board 101). In a specific 
implementation, each resonator signal is amplified and mixed with a square wave 
having the same frequency as the excitation signal, but is 1 80 degrees out of phase, 
15 to obtain a rectified version of each resonator signal. The rectified signal is then sent 
through a low pass filter to obtain the average voltage. The resulting average 
voltages are then sent through an analog-to-digital converter to the application 
software The application software determines the x and y position of the object 
based on the relative average voltages of the resonator signals on the reception coils. 
2 0 The object's position along each reception coil affects the relative values of the 
average voltages. For example, if a phasor vector is formed from the sin average 
voltage and the cos average voltage of the x direction reception coils, the Phaser's 
angle correlates with the object's x position relative to the x direction reception coils. 
In other words the angle of the phasor vector is directly related to the x position of 
2 5 the object. Likewise, the y position can be determined from the sin and cos average 
voltages of the y direction reception coils. The phasors' amplitude corresponds to the 
object's z position. 

The application software 112 analyzes the detected signals to determine 
positions and identifications of the various objects of the detection system 103, as 
30 well as to provide data content. The driver 114 is arranged to facilitate 
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communication between the application software 112 and the I/O board 108. The 
positional and/or identity data may be used by the apphcation program to generate an 
audiovisual program (e.g., a computer game) based on the detected position signals 
and corresponding object identities This data may also be used to access remote 
5 data to enhance the audiovisual program. For example, the positional and/or identity 
data is passed to the framework software 120, which then accesses remote data 
content, sends data to a remote site, or initiates a communication session between the 
local site and a remote site. In one embodiment, the framework software 120 
includes web browsing capabilities, email functions, and file transfer mechanisms. 
10 Preferably, the framework software 120 is separate from the application software 1 12 
to allow easy expansion of on-line capabilities and compatibility maintenance. Of 
course, the features of the application and framework software may be integrated into 
a single software module. AJternatively, the application and/or framework may be 
implemented with hardware and/or software. 

15 The above detection process is repeated for each frequency value. That is, 

the application software initiates an excitation signal at a second frequency value that 
corresponds to a next object; initiates an excitation signal at a third frequency value 
that corresponds to a next object; etc. The frequency values are sequentially used to 
detect the positions of each object. 

20 The detected signal may include positional data regarding the resonator 

circuits of one or more physical object(s) of the detection system 103, such as the 
objects, interactive devices, etc. That is, positions of one or more physical object(s) 
may be ascertained by analyzing the detected signal. In one embodiment, the 
positional data includes six degrees of position states: x, y, and z position, rotational 

2 5 angle, tilt, and yaw. One example of an electromagnetic sensing system that senses 
six degrees of positional data is described in the above referenced PCT patent 
application Of course, the positional data may include any subset of these six 
degrees of position states. 

Of course, it should be recognized that other detection systems may be used, 
30 such as optical sensors or electronic sensors. For example, a camera based detection 
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system may be implemented In this implementation, the tools and objects include 
visually detectable markers, such as barcodes, and a camera is mounted for detecting 
such makers. In one specific implementation, each tool or object has a unique 
barcode. The camera sends its detected images to an image processor which then 
analyzes the image to determine positions and/or identities of the object or tool 
markers Several embodiments of camera based systems and detection techniques 
are further described in the above referenced U.S. Patent Numbers 5,953,686 and 
6,047,249 and U.S. Application Numbers 09/017,450 and 09/103,265. 

To detect and identify the objects of the detection system 103, at least some 
of the objects v^ill include at least one delectable marker. When the markers are 
placed or moved within the platform 104 (e.g., by placing an object on the platform 
and moving the object over different portion of the platform), the positions of the 
markers may be sensed by the detection mechanism. Preferably, the markers also 
identify the object. That is, the detection mechanism is able to distinguish between 
the different markers of the different tools and objects. In the above described 
electromagnetic detection system, each object includes a unique resonator circuit that 
resonates at a unique frequency. Alternatively, some or all of the objects may 
include the same resonator circuit, but the resonator circuit is placed at a different z 
position of the object. In this embodiment, each object is identified when it is placed 
one at a time onto the platform. Thus, each object will have a different detectable z 
position when it is placed on the platform The z position can then be used to 
distinguish between the objects. Several mechanisms for identifying resonator type 
objects are also described in above referenced U.S. Patent Application Number 
09/144,951. In an optical camera based detection system, each object may include a 
unique detectable visual marker, such as a barcode. Several mechanisms for 
identifying visually marked objects are described further in above referenced U.S. 
Patent Numbers 5,953,686 and 6,047,249 and U.S. Application Numbers 09/017,450 
and 09/103,265. 

Although the computer system 1 10 is shown as being a separate component 
from the detection system 103, of course, the computer system 1 10 may be integrated 
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with the detection system 103. Additionally, it should be understood that other type 
of interfaces or computer systems may be used, such as a television system or set top 
box (See Fig 4 and associated description below) Also, the I/O board 108 may be a 
separate device from the detection system 103 and computer system 1 10. 

5 The interactive system 100 may include any number and kind of objects. The 

objects are used to interact with portions of the detection system 103 (e.g., platform 
104) so as to initiate content delivery, e.g., Internet usage The interactive system 
may also mclude interactive devices, which are defined as objects that have more 
than one state For example, an interactive device may be in the form of a button, 
1 0 switch, or knob. 

Some of the objects may also include an interactive device 107, such as a 
button The detection system 103 may also include one or more fixed interactive 
devices (eg., 109), such as a switch or button, (i e., the device has a fixed resting 
position) In short, the objects and interactive devices may be fixed or movable 
within the detection system 103. An interactive device may be associated with a 
particular object or may be an independent object \yithin the detection system 103. 
As described above, each object and interactive device may be configured with a 
resonator that resonates at a distinct frequency. Of course, if an interactive device is 
associated with an object, the pair may have a single resonator 

Any suitable technique may be implemented for obtaining positional and/or 
identity data from the detection system and using such data to access content 
(remotely or locally). FIG. 2 is a flowchan illustrating a process 200 of interfacing 
with a detection system, such as the system in FIG. 1, in accordance with one 
embodiment of the present invention. Initially, a detection system is provided in 
operation 202, In operation 204 portions of the detection system (e.g., the platform 
104) are scanned to extract data. The extracted data includes information regarding 
the physical objects (e.g., objects and interactive devices) of the detection system. 
For example, the extracted data includes positions and orientation for each physical 
object relative to each other and/or relative to the platform. Also, the extracted data 
may include identifiers associated with each physical object. Alternatively, the 
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identity of an object is already known and used to detect positional data from the 
identified object. 

In a detection system that includes electromagnetic sensing technology, the 
data is extracted by initiating an excitation signal at a predetermined frequency on an 
5 antenna. The excitation signal is then stopped, and it is determined whether a 
detected signal is present on the antenna from one or more resonators within the 
detection system. The detected signal may include positional information related to 
the responding resonators of the detection system. The predetermined frequency 
may correspond to an identity of a responding resonator. A succession of excitation 
10 signals having different predetermined frequencies may be initiated on the antenna to 
detect the presence and position of multiple physical objects having different 
resonating frequencies. The identity of each object may correspond to the object's 
resonance frequency or the position (e.g.. z position) of the resonator of the object. 

After the data is extracted from portions of the platform, the extracted data is 
1 5 interpreted in operation 206. For example, the extracted data may be analyzed to 
determine the relative positions and identities of various physical objects within the 
detection system. The extracted data may also be analyzed to determined the relative 
positions and identities of the detected physical objects and various regions of the 
platform. Operation 206 is described in fijrther below in reference to FIG. 3. 

2° -^er the data is interpreted, it is then determined whether remote data 

content is to be accessed in operation 208. This determination is based on the results 
from the interpretation operation 206 and also depends on the particular requirements 
of the application For example, particular remote content may be accessed when a 
particular object is positioned within a particular area of the platform. In another 

2 5 example, particular remote content may be accessed when any object is positioned 
within such area. Particular remote data may be accessed when two or more objects 
have a particular orientation relative to each other and/or a particular area of the 
platform. 

If it is determined that remote content is required, the remote data is then 
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accessed in operation 210. For example, a background file transfer may be 
performed, where data content is downloaded (e.g., through a script file) so that the 
application may later use by the application. In another implementation, additional 
game character costumes or adventure worlds may be downloaded to specific game 
directories within the game structure. Such game directories are accessible by the 
game application. Content may also be posted (instead of access remote data 
content) to a remote site. In specific examples, user-created content such as movies, 
team logos, treasures, or game content can be posted to a remote server, where it is 
screened and then made accessible to other users. 

If remote data is not to accessed, operation 210 is skipped. The application is 
then executed based on the accessed data (if any) in operation 212. For example, an 
audiovisual program is run that is based on the extracted data and any previously 
extracted and recorded data. For example, the relative position of regions of the 
platform and the previous physical object positions may have been previously 
recorded and thereby accessed to generate a new audiovisual program. 

The execution of the application may include merely one visual frame or may 
include one frame of a sequence of frames from a video. For example, the operation 
of executing the application may initiate a continuous audiovisual sequence (e.g., a 
Quicktime movie) or merely continue at a particular point within the movie. By way 
of another example, an audiovisual program may be executed that only has one 
frame. By way of another example, the audiovisual program may include an 
interactive game, wherein the player is directed to perform certain tasks with the 
physical objects. 

After the application is initiated or continued, the extracted data information 
is recorded in operation 214. After the extracted data is recorded, it is determined 
whether a shutdown of the interactive system has been initiated in operation 216. 
This may be accomplished in any suitable manner For example, when the all of the 
physical objects are removed from the platform, this may be detected and interpreted 
as a shutdown Alternatively, the detection system may include an on/off switch. 
Otherwise, the application may simply be ended {e,g,, with an exit command). If a 
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shutdown has not been initiated, the process returns to operation 204, where portions 
of the platform are scanned once again to extract data. After portions of the platform 
are scanned once again, operations 204 through 214 are repeated and may use the 
previously recorded data. When shutdown is initiated, the process 200 ends until it is 
5 initiated again (e.g., when physical objects are placed on the platform or the 
detection system is switched on). 

FIG. 3 is a flow chart illustrating the operation 206 of FIG. 2 of interpreting 
the extracted data in accordance with one embodiment of the present invention. 
Initially, the positions of selected physical objects that are present are determined in 
10 operation 302. The detection system may be designed such that physical objects may 
be placed on the platform one at a time, or so that two or more physical objects may 
be placed simultaneously on the platform. The identifiers of the selected physical 
objects are determined in operation 304 (e.g., by correlating the selected object's 
resonator frequency with an identity). 

It is then determined whether the selected physical objects are associated with 
a region or '*hot zone" on the platform in operation 306. That is, when a physical 
object is placed within a hot zone, a corresponding audiovisual application is 
generated. A hot zone is defined as a particular portion on the platform that is 
relevant to determining which content to select. In other words, the hot zones may 
be defined as regions where moveable objects may be placed or where fixed 
interactive devices, such as a fixed push button, are located. Alternatively, a hot 
zone may be defined relative to the sensor or antenna. The hot zones that are 
associated with a physical object are then identified in operation 308. For example, 
the hot zone associated with the physical object may be identified to determine what 
type of interaction will take place with the particular physical object. By way of 
specific example, a portion of the platform may be identified as being magnified (i.e., 
by the audiovisual program) when a magnifying type object is placed on it. In this 
case, a web page representative of such magnification may be presented to the user 
(e.g., in operation 212 of FIG. 2). 

The states of any interactive devices are then determined in operation 310. 
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The interactive devices may be in any suitable form for indicating a different state of 
a physical object. For example, a physical object may include a button which may be 
pushed in or let out. It is then determined whether the button is pushed in operation 
310. The process then returns to operation 212 of FIG. 2 where an application is 
5 executed that is based on the interpretation of the extracted data. 

It should be well understood to those skilled in the art that operations 302 
through operation 310 may be executed in any order that is appropriate for the 
particular application. For example, operation 302 may be executed subsequent to 
operation 304. Additionally, other information from the extracted data may be 
10 interpreted or not interpreted depending on the particular requirements of the 
particular application For example, if the detection system does not include any 
interactive devices, operation 310 is not required. By way of another example, an 
operation to determine the identification of the platform itself may be included in 
order to aid in choosing which application will be run in operation 212 of FIG. 2. 
15 For example, various platforms may be utilized with the detection system 103. The 
platforms may themselves include resonators with different resonating frequencies, 
differently positioned resonators having the same frequency, or different barcodes, 

FIG. 4 is a flow chart illustrating the operation 212 of FIG. 2 for executing an 
application program based on data regarding one or more physical objects in 
2 0 accordance with one embodiment of the present invention. Initially, it is determined 
whether there are any physical objects associated with any hot zones in operation 
402. If there are no physical objects associated with a hot zone, the process returns 
to operation 214 of FIG. 2, and the extracted data is recorded. 

If there are physical objects associated with a hot zone, a physical object is 
selected in operation 404. A physical object may be selected in any suitable manner. 
For example, a physical object which was placed upon the platform first may be 
selected first. A corresponding application program, such as a game, is then 
executed or continued in operation 406. The application program is based on the 
selected physical object, the physical object identifier, a state of any associated 
interactive device, and/or the associated hot zore identifier. In other words, the 
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application program may depend on which physical object is selected and where 
such physical object is positioned in relation to hot zone regions. For example, a 
second on-line player, in addition to the local user, is added to the game currently 
being executed for the user. In one embodiment, a second character, in addition to 
the user's first character, is displayed within the game displayed at the user's site. 
This second character is controlled by the off-line second player, while the user 
controls his own first character. 

After the application program segment runs its course, it is then determined 
whether there are more physical objects to select in operation 408. If there are more 
physical objects to select, it is selected in operation 404 Another application 
segment is executed continues in operation 406. That is. the application that is 
currently running is updated based on the positions of the selected physical object. 
For example, a third on-line player or a third web page relevant to the game story 
line is added to the game. However, if there are no more physical objects to select, 
the process returns to operation 214 of FIG. 2 and the extracted data is recorded for a 
next scan of the platform (e,g.. operation 204). Of course, each physical object does 
not have to be separately selected prior to each application update. Alternatively, the 
application program may simply be executed or updated based on selection of a// 
physical objects or a subset of the same within the detection system. 

FIG. 5 is a diagrammatic representation of a remote control system 500 for 
accessing content through a television 510 in accordance with an alternative 
embodiment of the present invention. As shown, the remote control system includes 
a detection system 103' in the form of a tablet-like controller and a computer system 
1 10' in the form of a set top box (or satellite receiver) 1 10. The tablet-like controller 
103* includes one or more physical objects 502 and may also include a pen device 
504. Content may be selected by moving one or more physical objects 502 to 
particular positions on the tablet 103' or by arranging two or more physical objects 
502 into a particular orientation relative to each other. Additionally, content may be 
selected via handwriting motions of the pen device 504. 

Positional and/or identity data is scanned from the detection system 103' and 
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transmitted to the set top box 508 by any suitable mechanisms, such as an JR 
(infrared radiation) link circuit or other wireless link within the detection system. 
This data is then analyzed by the set top box 508 to determine which content to 
access, for example, through a cable network interface 512. In the illustrated 
5 embodiment, the tablet 103' includes a sensing area 508 upon which physical objects 
are detectable and a inactive area 506 upon objects may be stored without being 
sensed. An overlay (not shown) may also be provided over the tablet to aid in 
content navigation. For example, the overlay may include a channel accessing area 
and subject classes for web content. The accessed content may include any suitable 
10 information, such as a cable, satellite, or network channel, a web page, graphical 
and/or text data files, etc. The accessed content is then displayed on the television 
510. 

The physical objects or disks 502 in this implementation, or the other 
implementations described above, may be sold by content providers wishing to 
1 5 promote particular programs or web sites. For example, the disks may be sent along 
with a customer's TV Guide, a promotional flyer, or made available at a supermarket 
or sent through the mail. Customers may then use the disk within their detection 
system (e.g., 103 of FIG. 1 or 103* of FIG. 5) to access the particular program or web 
site. 

2 0 The disks may be used to display content on the user's computer or television, 

store content on the user's computer system, or to setup a recording session of the 
promoted particular program. Other disks may be used to filter certain data content 
subject matter, (eg., a particular disk may correspond to cartoons only), to control 
access for children or to setup subject preferences for others. Other disks may 
25 indicate a maximum viewing time for web surfing or television watching. When a 
timing disk is placed within the detection system, content viewing time is limited to 
the defined maximum time of the disk. In other embodiments, specific disks may 
each correspond to a particular web browsing function, such as scrolling or selecting 
a link. 

3 0 Regardless of interactive system's configuration, it may employ one or more 
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memories or memory modules configured to store data, program instructions for the 
accessing remote data content and/or detection and data analysis mechanisms 
described herein. The program instructions may control the operation of an 
operating system and/or one or more applications, for example. The memory or 
5 memories may also be configured to store positional data, identity data, accessed 
data content, etc. 

Because such information and program instructions may be employed to 
implement the systems/methods described herein, the present invention relates to 
machine readable media that include program instructions, state information, etc. for 

10 performing various operations described herein Examples of machine-readable 
media include, but are not limited to, magnetic media such as hard disks, floppy 
disks, and magnetic tape; optical media such as CD-ROM disks; magneto-optical 
media such as floptical disks; and hardware devices that are specially configured to 
store and perform program instructions, such as read-only memory devices (ROM) 

15 and random access memory (RAM). The invention may also be embodied in a 
carrier wave travelling over an appropriate medium such as airwaves, optical lines, 
electric lines, etc Examples of program instructions include both machine code, 
such as produced by a compiler, and files containing higher level code that may be 
executed by the computer using an interpreter. 

-0 Although the foregoing invention has been described in some detail for 

purposes of clarity of understanding, it will be apparent that certain changes and 
modifications may be practiced within the scope of the appended claims. Therefore, 
the described embodiments should be taken as illustrative and not restrictive, and the 
invention should not be limited to the details given herein but should be defined by 

:5 the following claims and their fijll scope of equivalents. For example, 
communication over a wide area network (£^.^.. Internet) may be initiated based on 
any combination of detected inputs from a plurality of objects (e,g,, position, 
identity, states of interactive devices, etc.). Additionally, the communication may be 
based on detected states of a plurality of objects over a period of time {e,g., 

0 communication is based on a gesture movement). Additionally, communication may 
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simply be based on a cuirent orientation of a single object or a current selectable 
state (e.g., temperature, pressure, voltage, current, velocity, etc.) of a single object. 
By way of a final example, specific communication types (e.g,, game content 
downloading or posting, web page presentation, chat session initiation, initiating an 
email session, video and/or audio streaming, or video conferencing) may be initiated 
by any type of interaction with one or more objects. 

Although the foregoing invention has been described in some detail for 
purposes of clarity of understanding, it will be apparent that certain changes and 
modifications may be practiced within the scope of the appended claims. 
Accordingly, the present embodiments are to be considered as illustrative and not 
restrictive, and the invention is not to be limited to the details given herein, but may 
be modified within the scope and equivalents of the appended claims. 
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CLAIMS 

Whar IS claimed is: 

1. An interactive system associated with a first computer for 
communicating with a second computer over a data network, the interactive system 
5 comprising: 

a detection system having a plurality of physical objects and a sensor, 
wherein the physical objects have a plurality of states that are each detectable by the 
sensor, and 

a computer readable medium having programming instructions for 
0 communicating with the detection system and the second computer, wherein 
communicating with the second computer is based on a combination of two or more 
states of the physical objects detected. 

2. An interactive system as recited in claim 1, wherein the combination of 
two or more states are selected from a group consisting of one or more positions of 

> one or more of the physical objects, an identity of one or more physical objects, and 
a current setting of a plurality of user selectable states, other than a position state, for 
one or more physical objects. 

3. An interactive system as recited in claim 2, wherein the one or more 
positions of the one or more physical objects are relative to each other. 

4. An interactive system as recited in claim 2, wherein the one or more 
positions of the one or more physical objects are relative to the sensor. 

5. An interactive system as recited in claim 2, wherein the one or more 
positions each includes an x, y, and z position of the one or more physical objects. 

6. An interactive system as recited in claim 1, wherein the detection 
system is implemented with an electromagnetic detection technology, each physical 
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object having a resonator circuit that is activated by an excitation signal and the 
sensor being configured to detect a resonating signal from the resonator circuit. 

7. An interactive system as recited in claim 6, wherein each resonator 
circuit of each physical object has a unique resonating frequency. 

5 8. An interactive system as recited in claim 6, wherein at least two of the 

each resonator circuit of each physical object has a same resonating frequency but 
the each resonator circuit being placed at a different position relative to its physical 
object. 

9 An interactive system as recited in claim 1, wherein communicating 
1 0 with the second computer is selected from a group consisting of a hypertext transfer, 

a file transfer, and an email transfer. 

10 An interactive system as recited in claim 1, wherein communicating 
with the second computer is performed via a wide area network. 

II. An interactive system associated with a first computer for 
15 communicating with a second computer over a wide area network, the interactive 
system comprising: 

a detection system having a physical object and a sensor, wherein the 
physical object has a plurality of states that are each detectable by the sensor and that 
are states other than position states of the physical object relative to the sensor; and 

2 0 a computer readable medium having programming instructions for 

communicating with the detection system via the first computer and for 
communicating with the second computer via a wide area network, wherein 
communicating with the second computer network is based on a current state of the 
physical object. 

2 5 12. An interactive system as recited in claim 11, wherein communicating 

with the second computer is further based on detected states of the physical object 
over time. 
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13. An interactive system as recited in claim 11, wherein the detection of 
the current state of the physical object is based on a current position of the physical 
object that is detected by the sensor. 

14. An interactive system as recited in claim 11, wherein the physical 
5 object includes a rotatable knob and the states of the physical object are a plurality of 

rotational positions. 

15. An interactive system as recited in claim 11, wherein the physical 
object includes a movable lever and the states of the physical object are a plurality of 
lever positions. 

^0 16. An interactive system as recited in claim 11, wherein the physical 

object includes a sensor and the states of the physical object are a plurality of sensor 
states. 

17 An interactive system as recited in claim 16, wherein the sensor is a 
temperature sensor and the sensor states are temperatures. 

1^ 18. An interactive system as recited in claim 16, wherein the sensor is a 

voltage sensor and the sensor states are voltage values. 

19. An interactive system as recited in claim 16, wherein the sensor is a 
current sensor and the sensor states are current values. 

20. An interactive system as recited in claim 16, wherein the sensor is a 
20 pressure sensor and the sensor states are pressure values. 

21. An interactive system as recited in claim 16, wherein the sensor is a 
velocity sensor and the sensor states are velocity values. 

22. An interactive system as recited in claim 11, wherein the computer 
readable medium is selected from a group consisting of one or more floppy disks, 
one or more optical media, one or more magneto-optical media, a carrier wave, and a 
configured hardware device. 
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23. An interactive system as recited in claim 11, wherein communicating 
with the second computer includes tasks selected from a group consisting of 
downloading data to the first computer, posting data to the second computer, and 
setting up a communication session between the first computer and the second 
5 computer. 

24 An interactive system as recited in claim II, wherein the detection 
system is implemented with an electromagnetic detection technology, the physical 
object having a resonator circuit that is activated by an excitation signal and the 
sensor being configured to detect a resonating signal from the resonator circuit. 

1^ 25 An interactive system as recited in claim 11, wherein communicating 

with the second computer is selected from a group consisting of a hypertext transfer, 
a file transfer, and an email transfer. 

26 An interactive system as recited in claim II, wherein the wide area 
network is the Internet. 

27. An interactive system associated with a first computer for 
communicating with a second computer over a wide area network, the interactive 
system comprising: 

a detection system having a physical object and a sensor, wherein the 
physical object has a plurality of orientation positions that are each detectable by the 
2 0 sensor, and 

a computer readable medium having programming instructions for 
communicating with the detection system via the first computer and for 
communicating with the second computer via a wide area network, wherein 
communicating with the second computer is based on a current orientation of the 
2 5 physical object. 

28. An interactive system as recited in claim 27, wherein the orientation 
positions includes a plurality of rotational positions 
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29. An interactive system as recited in claim 27, wherein the orientation 
positions includes a plurality of pitch positions. 

30. An interactive system as recited in claim 27, wherein the orientation 
positions includes a plurality of yaw positions. 

31. An interactive system as recited in claim 27, wherein the detection 
system is implemented with an electromagnetic detection technology, the physical 
object having a resonator circuit that is activated by an excitation signal and the 
sensor being configured to detect a resonating signal from the resonator circuit. 

32 An interactive system as recited in claim 27, wherein communicating 
with the second computer is selected from a group consisting of a hypertext transfer, 
a file transfer, and an email transfer. 

33. An interactive system as recited in claim 27, wherein the wide area 
network is the Internet. 

34. An interactive system associated with a first computer for 
communicating with a second computer over a data network, the interactive system 
comprising: 

a detection system having a plurality of physical objects and a sensor, 
wherein the physical objects have a plurality of positions that are each detectable by 
the sensor; and 

a computer readable medium having programming instructions for 
communicating with the detection system and the second computer, wherein 
communicating with the second computer is based on one or more positions of the 
physical objects, wherein the positions are detected by the sensor and are non- 
discrete. 

35. An interactive system as recited in claim 34, wherein the positions of 
the physical objects are relative to the sensor. 
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36. An interactive system as recited in claim 35, wherein the positions 
includes an x, y, and z position of the physical objects. 

37. An interactive system as recited in claim 34, wherein the detection 
system is implemented with an electromagnetic detection technology, each physical 

5 object having a resonator circuit that is activated by an excitation signal and the 
sensor being configured to detect a resonating signal from the resonator circuit. 

38 An interactive system as recited in claim 37, wherein each resonator 
circuit of each physical object has a unique resonating frequency. 

39 An mteractive system as recited in claim 37, wherein at least two of the 
10 each resonator circuit of each physical object has a same resonating frequency but 

the each resonator circuit being placed at a different position relative to its physical 
object. 

40. An interactive system as recited in claim 34, wherein communicating 
with the second computer is selected from a group consisting of a hypertext transfer, 

15 a file transfer, and an email transfer. 

41. An interactive system as recited in claim 34, wherein communicating 
with the second computer is performed via a wide area network. 

42. An interactive system associated with a first computer for 
communicating with a second computer over a data network, the interactive system 

2 0 comprising: 

a detection system having a plurality of physical objects and a sensor, 
wherein interaction data associated with the physical objects is detectable by the 
sensor; and 

a computer readable medium having programming instructions for 
2 5 communicating with the detection system and the second computer, wherein 
communicating with the second computer is based on detected interaction data 
associated with one or more of the physical objects, wherein the communicating is 
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selected from a group consisting of downloading game content from the second 
computer to the first computer, posting game content from the first computer to the 
second computer, displaying a web page at the first computer, initiating a chat 
session between the first and second computers, sending or receiving an email 
message to or from the second computer, setting up a video conference, and 
streaming video or audio to the first computer from the second computer or to the 
second computer from the first computer. 

43. An interactive system as recited in claim 42, wherein the computer 
code for communicating with the second computer is a framework software 
component and wherein the computer readable medium ftirther comprises an 
application software component having computer code for executing an application 
related to the theme, the framework being a separate component from the application 
software. 

44. An interactive system as recited in claim 43, wherein the framework is 
configured to make a remote content segment available to the application software 
for later use by the application software. 

45. An interactive system as recited in claim 44, wherein the remote 
content segment is downloaded into a directory associated with the application 
software. 

46. An interactive system as recited in claim 42, wherein the detection 
system is implemented with an electromagnetic detection technology, each physical 
object having a resonator circuit that is activated by an excitation signal and the 
sensor being configured to detect a resonating signal from the resonator circuit. 

47. An interactive system as recited in claim 46, wherein each resonator 
circuit of each physical object has a unique resonating frequency. 

48. An interactive system as recited in claim 46, wherein at least two of the 
each resonator circuit of each physical object has a same resonating frequency but 
the each resonator circuit being placed at a different position relative to its physical 
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object. 

49. An interactive system as recited in claim 42, wherein communicating 
with the second computer is selected from a group consisting of a hypertext transfer, 
a file transfer, and an email transfer. 

5 50. An interactive system as recited in claim 42, wherein the detection 

system is in the form of a tablet-like controller for a television system and further 
comprises a pen device for entering handwriting motions, the communicating with 
the second computer being further based on handwriting motions of the pen device, 

51. An interactive system as recited in claim 50, the detection system 
10 further comprising an overlay that is placed over the sensor to facilitate navigation 

through data content on the second computer by moving the physical objects over 
particular portions of the overlay and sensor to thereby cause remote content to be 
accessed that is associated with particular overlay portions. 

52. An interactive system as recited in claim 51, wherein particular 
1 5 portions of the overlay indicate particular categories of web content. 

53 An interactive system as recited in claim 42, the computer readable 
medium further comprises computer code for displaying the downloaded game 
content on a display device. 

54. An interactive system as recited in claim 42, wherein the 
2 0 communicating with the second computer is performed via a computer network. 

55. An interactive system as recited in claim 42, wherein the 
communicating with the second computer is performed via a cable television 
network or a satellite television network. 

56. An interactive system as recited in claim 42, the computer readable 
25 medium further comprising computer code for filtering a first remote content 

segment from the second computer from being presented to the first computer based 
on the interaction data. 
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57. A computer implemented method associated with a first computer for 
communicating with a second computer over a data network, the method comprising; 

in a detection system having a plurality of physical objects and a sensor, 
wherein the physical objects have a plurality of positions, detecting with the sensor at 
5 least one of the positions of at least one of the physical objects; and 

communicating with the second computer based on one or more positions of 
the physical objects, wherein the positions are detected by the sensor and are non- 
discrete. 

58 A computer implemented method associated with a first computer for 
1 0 communicating with a second computer over a data network, the method comprising: 

in a detection system having a plurality of physicaf objects and a sensor, 
detecting with the sensor interaction data associated with the physical objects; and 

communicating with the detection system and the second computer, wherein 
communicating with the second computer is based on detected interaction data 

15 associated with one or more of the physical objects, wherein the communicating is 
selected from a group consisting of downloading game content from the second 
computer to the first computer, posting game content from the first computer to the 
second computer, displaying a web page at the first computer, initiating a chat 
session between the first and second computers, sending or receiving an email 

20 message to or from the second computer, setting up a video conference, and 
streaming video or audio to the first computer from the second computer or to the 
second computer from the first computer. 
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